PNL: Efficient long-range dependencies extraction with pyramid non-local module for action recognition

Long-range spatiotemporal dependencies capturing plays an essential role in improving video features for action recognition. The previously introduced non-local block, inspired by the means, is designed to address this challenge and have shown excellent performance. However, block brings significant increase computation cost original network. It also lacks ability model regional correlations videos. To above limitations, we propose Pyramid Non-Local (PNL) module, which extends incorporating at multiple scales through a pyramid structured module. This extension upscales effectiveness of attending interaction between different regions. Empirical results prove efficiency our PNL achieves state-of-the-art performance 83.09% on Mini-Kinetics dataset, with decreased computational compared block.